from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.700	0.058	0.0	0.019	-1	5	0.735	3.202	0.183	5.839	0.335	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.700	0.058	0.0	0.019	-1	5	0.735	0.032	0.003	591.749	56.495	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.309	0.009	0.0	0.309	-1	5	1.000	3.202	0.183	0.096	0.006	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.309	0.009	0.0	0.309	-1	5	1.000	0.032	0.003	9.763	0.977	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.437	0.019	0.0	0.018	-1	1	0.675	1.991	0.027	9.261	0.127	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.437	0.019	0.0	0.018	-1	1	0.675	0.029	0.003	631.417	60.966	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.312	0.010	0.0	0.312	-1	1	1.000	1.991	0.027	0.157	0.005	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.312	0.010	0.0	0.312	-1	1	1.000	0.029	0.003	10.701	1.087	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.346	0.045	0.0	0.018	1	5	0.735	2.268	0.008	8.089	0.033	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.346	0.045	0.0	0.018	1	5	0.735	0.027	0.000	684.643	6.047	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.008	0.0	0.310	1	5	1.000	2.268	0.008	0.137	0.004	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.008	0.0	0.310	1	5	1.000	0.027	0.000	11.574	0.326	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.789	0.010	0.0	0.019	1	100	0.845	2.281	0.010	8.238	0.037	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.789	0.010	0.0	0.019	1	100	0.845	0.031	0.009	613.952	174.495	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.004	0.0	0.310	1	100	1.000	2.281	0.010	0.136	0.002	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.004	0.0	0.310	1	100	1.000	0.031	0.009	10.129	2.881	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.717	0.059	0.0	0.019	1	1	0.675	1.242	0.005	15.072	0.077	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.717	0.059	0.0	0.019	1	1	0.675	0.026	0.000	731.248	9.088	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.005	0.0	0.310	1	1	1.000	1.242	0.005	0.250	0.004	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.310	0.005	0.0	0.310	1	1	1.000	0.026	0.000	12.124	0.232	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.496	0.133	0.0	0.018	-1	100	0.845	3.045	0.031	6.074	0.076	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.496	0.133	0.0	0.018	-1	100	0.845	0.031	0.003	602.846	58.437	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.309	0.004	0.0	0.309	-1	100	1.000	3.045	0.031	0.101	0.002	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.309	0.004	0.0	0.309	-1	100	1.000	0.031	0.003	10.066	0.981	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.610	0.007	0.0	0.004	-1	5	0.889	2.771	0.038	1.303	0.018	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.610	0.007	0.0	0.004	-1	5	0.889	0.010	0.004	361.773	153.015	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.004	0.0	0.243	-1	5	1.000	2.771	0.038	0.088	0.002	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.243	0.004	0.0	0.243	-1	5	1.000	0.010	0.004	24.312	10.291	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.629	0.051	0.0	0.004	-1	1	0.851	1.796	0.028	2.021	0.043	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.629	0.051	0.0	0.004	-1	1	0.851	0.010	0.005	381.927	214.380	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.005	0.0	0.242	-1	1	1.000	1.796	0.028	0.135	0.003	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.005	0.0	0.242	-1	1	1.000	0.010	0.005	25.429	14.277	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.620	0.019	0.0	0.004	1	5	0.889	2.055	0.002	1.761	0.009	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.620	0.019	0.0	0.004	1	5	0.889	0.003	0.000	1384.352	113.318	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	1	5	1.000	2.055	0.002	0.117	0.002	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.005	0.0	0.240	1	5	1.000	0.003	0.000	91.870	7.702	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.663	0.010	0.0	0.004	1	100	0.905	2.068	0.003	1.772	0.006	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.663	0.010	0.0	0.004	1	100	0.905	0.003	0.000	1386.575	118.291	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	100	1.000	2.068	0.003	0.117	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	100	1.000	0.003	0.000	91.205	8.001	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.609	0.006	0.0	0.004	1	1	0.851	1.135	0.005	3.180	0.016	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.609	0.006	0.0	0.004	1	1	0.851	0.002	0.000	1918.099	254.755	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	1	1.000	1.135	0.005	0.212	0.004	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.005	0.0	0.241	1	1	1.000	0.002	0.000	127.973	17.168	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.665	0.007	0.0	0.004	-1	100	0.905	2.800	0.035	1.309	0.016	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.665	0.007	0.0	0.004	-1	100	0.905	0.011	0.006	342.946	180.527	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	-1	100	1.000	2.800	0.035	0.086	0.002	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	-1	100	1.000	0.011	0.006	22.545	11.874	See	See

KNeighborsClassifier_kd_tree

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.163	0.000	0.0	0.120	1	100	0.975	5.585	0.012	21.514	0.048	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	120.163	0.000	0.0	0.120	1	100	0.975	0.010	0.000	11932.260	500.955	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.587	0.099	0.0	2.587	1	100	1.000	5.585	0.012	0.463	0.018	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.587	0.099	0.0	2.587	1	100	1.000	0.010	0.000	256.917	14.589	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	116.350	0.000	0.0	0.116	-1	1	0.962	0.506	0.007	229.867	3.392	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	116.350	0.000	0.0	0.116	-1	1	0.962	0.005	0.000	21534.826	1159.926	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.583	0.090	0.0	2.583	-1	1	1.000	0.506	0.007	5.104	0.194	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.583	0.090	0.0	2.583	-1	1	1.000	0.005	0.000	478.135	30.700	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.995	0.000	0.0	0.126	-1	5	0.975	0.960	0.011	131.293	1.484	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.995	0.000	0.0	0.126	-1	5	0.975	0.006	0.001	21458.442	2762.264	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.598	0.077	0.0	2.598	-1	5	1.000	0.960	0.011	2.707	0.086	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.598	0.077	0.0	2.598	-1	5	1.000	0.006	0.001	442.489	58.445	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.794	0.000	0.0	0.127	1	5	0.975	1.643	0.008	77.172	0.374	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.794	0.000	0.0	0.127	1	5	0.975	0.004	0.001	31130.428	5098.098	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.579	0.047	0.0	2.579	1	5	1.000	1.643	0.008	1.570	0.029	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.579	0.047	0.0	2.579	1	5	1.000	0.004	0.001	633.179	104.328	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.144	0.000	0.0	0.131	-1	100	0.975	3.200	0.011	40.979	0.140	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.144	0.000	0.0	0.131	-1	100	0.975	0.012	0.001	11346.502	683.989	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.602	0.100	0.0	2.602	-1	100	1.000	3.200	0.011	0.813	0.031	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.602	0.100	0.0	2.602	-1	100	1.000	0.012	0.001	225.096	16.098	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.870	0.000	0.0	0.137	1	1	0.962	0.860	0.004	159.108	0.787	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.870	0.000	0.0	0.137	1	1	0.962	0.003	0.001	54021.600	10823.757	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.588	0.053	0.0	2.588	1	1	1.000	0.860	0.004	3.008	0.063	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.588	0.053	0.0	2.588	1	1	1.000	0.003	0.001	1021.351	205.687	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.001	0.0	0.000	1	100	0.904	0.035	0.000	1.769	0.020	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.001	0.0	0.000	1	100	0.904	0.001	0.000	99.894	24.047	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	1	100	1.000	0.035	0.000	0.157	0.016	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	1	100	1.000	0.001	0.000	8.884	2.325	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	1	0.877	0.022	0.001	1.694	0.054	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	1	0.877	0.003	0.001	12.504	3.204	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.022	0.001	0.232	0.010	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	1	1.000	0.003	0.001	1.709	0.441	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	5	0.912	0.024	0.001	1.597	0.073	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	-1	5	0.912	0.003	0.000	12.684	1.762	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.024	0.001	0.218	0.011	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.003	0.000	1.729	0.244	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	5	0.912	0.021	0.000	1.853	0.012	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.000	0.0	0.000	1	5	0.912	0.001	0.000	59.940	19.127	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.021	0.000	0.252	0.008	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.139	2.608	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.904	0.037	0.005	1.652	0.229	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.061	0.000	0.0	0.000	-1	100	0.904	0.002	0.000	28.388	5.959	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.037	0.005	0.142	0.020	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	100	1.000	0.002	0.000	2.433	0.515	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	1	1	0.877	0.019	0.000	1.967	0.033	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	1	1	0.877	0.001	0.000	63.377	16.954	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.019	0.000	0.271	0.010	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	8.725	2.350	See	See

HistGradientBoostingClassifier_best

onnx (1.9.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.479	0.015	0.002	0.000	0.81	0.121	0.008	3.951	0.282	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.479	0.015	0.002	0.000	0.81	0.017	0.001	28.169	1.418	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.391	0.010	0.000	0.391	1.00	0.121	0.008	3.225	0.222	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.391	0.010	0.000	0.391	1.00	0.017	0.001	22.990	1.072	See	See